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SUMMARY 

The  convergence  rale  for  a  number  of  iterative  procedures  for  the  method  of  centres , 
was  studied  in  connection  with  the  investigation  of  methods  for  extending  the  applicability 
of  flight  directors.  By  the  use  of  the  Kuhn-Tucker  conditions  and  the  duality  properties 
for  convex  programming  problems ,  it  was  shown  that  the  augmented  cost  function ,  arising 
in  this  method \  has  a  second  order  zero  at  the  optimum  point.  From  this  flows  the  results: 
that  the  Stoha  and  Morrison  iteration  procedures  are  linearly  convergent ;  the  tangent 
iteration  procedure  is  quadratically  convergent;  and  two  interpolation  polynomial 
iteration  procedures  proposed  by  the  author  to  overcome  the  deficiencies  of  the  tangent 
method  away  from  the  optimum  point  are  super ’linearly  convergent  and  are  thus  worthy 
of  further  investigation . 
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1.  INTRODUCTION 


The  range  of  applicability  of  flight  directors  to  situations  involving  large  manoeuvres  and 
disturbances  is  capable  of  being  extended  by  the  incorporation  of  predictive  methods  in  digital 
avionic  and  display  systems.  This  involves  the  determination  of  optimal  controls  in  the  presence 
of  physical  and  operational  constraints,  and  alternative  numerical  algorithms  for  this  purpose 
are  being  investigated-  In  this  note  the  speed  of  convergence  for  the  method  of  centres  optimi¬ 
zation  algorithm,  which  can  handle  the  above  types  of  constraints,  is  examined  to  determine 
its  suitability  for  rapid  computations. 

The  method  of  centres  algorithm  is  a  well  established  technique  for  finding  the  minimum 
of  a  function  of  n  variables  subject  to  inequality  constraints.  Lootsma1  distinguishes  two  classes 
of  such  algorithms,  namely:  interior  and  exterior  points  methods.  For  the  former  methods 
extensive  theoretical  and  computational  studies,  of  their  convergence  properties  and  rates  of 
convergence,  have  been  made  by  Huard,2  Faure  and  Huard,3  Lootsma,5*4  and  Pironneau  and 
Polak.5  In  particular  Lootsma,1  and  Pironneau  and  Polak5  have  shown  that  the  algorithms 
proposed  by  Huard  are  linearly  convergent. 

Exterior  point  methods,  the  subject  of  this  report,  were  first  proposed  by  Kowalik,8  and 
have  been  studied  both  theoretically  and  computationally  by  Morrison,6  Kowalik  et  ai7  and 
Lootsma.1,4  Numerical  experience  with  a  number  of  these  methods  has  been  variable,  with 
Lootsma  reporting  disappointing  results  for  the  algorithm  considered  in  Reference  4,  while 
Staha  and  Himmelblau9  reported  very  favourably  on  their  application  of  Newton's  method  to 
the  exterior  point  method  of  centres.  Lootsma10  studied  the  theoretical  rates  of  convergence  for 
a  number  of  exterior  point  algorithms  including  the  one  proposed  by  Morrison,6  the  tangent 
algorithm  discussed  by  Kowalik  et  ai 7  and  the  Newton  method  discussed  by  Staha.9  He  con¬ 
cluded  that  the  Morrison  and  Staha  methods  are  linearly  convergent  while  the  tangent  method 
is  supcr-linearly  convergent.  Numerical  experience,  however,  has  shown  that  Staha's  method 
usually  converges  more  rapidly  than  the  tangent  method,  which  tends  to  overshoot  the  optimum 
solution.  This  apparent  paradox  between  the  Staha  and  tangent  methods  is  even  more  com¬ 
pounded  when  it  is  noted  that  their  iteration  formulae  only  differ  by  a  factor  of  one  half. 

In  this  report  we  investigate  the  convergence  behaviour  of  the  Morrison,  tangent,  and  Staha 
algorithms,  as  well  as  two  interpolation  polynomial  algorithms  proposed  by  the  author.  The 
approach  used,  which  differs  from  that  of  Lootsma,  not  only  confirms  his  results  for  the  Morrison 
and  Staha  methods,  but  also  gives  further  insight  into  the  reason  why  the  Staha  algorithm  is 
only  linearly  convergent,  yet  the  tangent  algorithm  is  super-linearly  convergent.  In  fact  we  are 
able  to  demonstrate  that  the  tangent  algorithm  is  quadraticaily  convergent  as  well  as  obtaining 
an  explicit  formula  for  its  asymptotic  error  constant.  In  addition  we  are  able  to  show  that  one 
of  the  proposed  algorithms  is  almost  quadraticaily  convergent.  This  appears  to  confirm  the 
limited  computational  experience  obtained  to  date  which  shows  that  its  convergence  rate  for 
test  problems  compares  very  favourably  with  the  tangent  method. 


2.  PROBLEM  DEFINITION  AND  PRELIMINARY  RESULTS 

Jn  this  report  we  shall  consider  exterior  point  method  of  centres  algorithms  for  solving  the 
following  constrained  minimization  problem: 

PROBLEM  MP.  Find  a  point  veC  where  C  -  {xe  Rn  (/#(*)  ^  0,  i  —  1, .  .  m}  such  that 

/«Lr)  min  fa  x).  (2.1) 

v*  r 


1 


We  shall  assume  that  the  functions /<(*)»  i  =  0, 1, . . m  mapping  R*  into  Rl  are  twice  con¬ 
tinuously  differentiable.  The  gradient  and  hessian  of  a  function  ft  at  Jc  e  R*  will  be  denoted  by 
/<*(£)  and  respectively.  The  gradient  will  be  considered  to  be  a  row  vector  and  will  be 

written  as 

(2.2) 

\  3*1  ixn ) 

while  the  kl' th  element  of  the  hessian  will  be  written  as 


i>Xk  2>JC I 


kj= 


(2.3) 


The  set  C,  defined  above,  will  be  referred  to  as  the  constraint  set  and  it  will  be  assumed 
that  it  is  compact  with  a  non-empty  interior.  Any  point  jc  g  C  will  be  referred  to  as  an  admissible 
solution  of  problem  MP.  From  the  compactness  assumption  on  Cand  the  fact  that /0  is  continuous 
it  follows  that  an  Jc  g  C  exists  satisfying  (2.1). 

Problem  MP  with  the  additional  assumption  that  the  functions  /(),  i  =  0, 1, . . m  are 
convex  on  Rn ,  will  be  termed  a  convex  programming  problem. 

In  considering  the  method  of  centres  we  introduce  the  augmented  cost  function  Qt(x)y 
dependant  on  the  parameter  t  e  Rl  and  the  variable  x  e  Rny  which  is  defined  in  terms  of  the 
functions /(jc),  i  =  0, 1, . . m  given  in  problem  MP  above.  This  function  is  defined  as 


Qt( x)  =  [max(0,/o(jc)  -  /)]*  +  £  [max(0,/,(jc))]2.  (2.4) 

i  =  i 

For  any  t  we  denote  the  point  xe  Rn  minimizing  Qt(x)  as  jc(f  )•  The  significance  of  this  definition 
of  Qt(x)  follows  from  LootsmaV  result  that,  under  appropriate  assumptions,  if  a  sequence  of 
parameters  tu  i  =  1,2,...  converges  to  v=  /(jc)  then  the  corresponding  sequence  of  points 
x(/<)  e  Rn ,  i  —  1,2,...  converges  to  jc.  In  the  sequel  we  will  study  the  rate  of  convergence  of 
several  algorithms  for  adjusting  the  parameter  t  so  that  the  sequence  of  points  (jc(/<)}  generated 
by  it  approaches  Jc  from  the  exterior  of  the  constraint  set  C. 

Before  proceeding  we  state  some  basic  results  which  will  be  used  in  later  sections. 


THEOREM  2.L  Suppose /(•)  e  Cl{Rn ;  R1),  i  =  0,  1, . . m  and  that  problem  MP  is  a  convex 
programming  problem.  In  addition  assume  that  the  interior  of  the  constraint  set  C  is  non-empty. 
Then  an  admissible  solution  Jc  is  a  minimum  solution  of  problem  MP  if  and  only  if  there  exists 
a  we/?*  such  that 


for  /  =  1, . . .,  and 

J 

Proof.  See  Lootsma  (Ref.  1,  p.  25). 


fi(x)  ^  0, 

(2.5) 

>  0, 

(2.6) 

Uif,(x)  0, 

(2.7) 

m 

+  2  =  0. 

(2.8) 

DEFINITION  2.1.  A  point  (jc,  m)  e  RH  x  Rm  is  termed  a  Kuhn-Tucker  point  of  problem  MP 
if  it  satisfies  the  conditions  (2.5)  to  (2.8). 

We  introduce  the  following  notation:  Let  /(*)  *=*  {#  |/<(jc)  =  0,  /  =  1, . . .,  m)  denote  the  set 
of  active  constraints.  The  derivative  of  the  left  hand  side  of  (2.8)  will  be  denoted  by 


D2(xy  u )  =/0*x(x)  +  £  &if<xx(x)*  (2.9) 

i  i 

Following  Lootsma  we  give  the  following  definition: 

DEFINITION  2.2.  A  Kuhn-Tucker  point  (Jc,  m)  of  problem  MP  satisfies  the  Jacobian  Unique¬ 
ness  Condition  if : 

(a)  ut  >  0  for  i  e  /(Jc), 

(b)  i  e  /(j r)  are  linearly  independent,  and 

(c)  for  every  non-zero  y  e  R*y  fig(x)y  =  0,  /  e  /(Jr)  implies  >'t/)2(jc,  u)y  >  0. 
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THEOREM  2.2.  Suppose  /<(*)  e  C^R*;  R1),  i  =  0, 1, . . m  and  there  exists  a  Kuhn-Tucker 
point  (x,  u)  satisfying  the  Jacobian  uniqueness  condition  then  the  point  x  is  an  isolated  local 
minimum  of  problem  MP  and  the  vector  u  is  uniquely  determined. 


Proof.  Sec  Lootsma  (Reference  1,  p.  17). 

We  now  introduce  the  dual  problem  to  problem  MP. 

PROBLEM  DP.  Suppose /,(•)  e  C*(R";  R1),  i  =  0, 1, . . m.  Find  *  and  u  e  Rw  such  that 


/^)  +  SW)=  max  (/o(*)  +  2  (2.10) 

/  =  1  <x,  if)  €  Y  \  i  =  1  / 

where 

(*,  u)  e  T  =  |(x,  u)  |  x  e  Rn,  u  e  Rm,/0*(x)  -b  2  u</<*(x)  =  0,  «<  >  0,  i  =  1, . . mj. 

We  define 


«)  =/o(*)  +  2  «</<(x) 


Wolfe11  has  proved  the  following  result: 


/  =  i 


(2.11) 


THEOREM  2.3.  Suppose /<(  )  e  CMR*;  R1),  /  =  0, 1, . . m  and  that  problem  MP  is  a  convex 
programming  problem.  If  Jc  is  a  minimum  of  problem  MP  and  satisfies  a  suitable  constraint 
qualification  condition  then  there  exists  a  u  e  Rw  such  that  (x,  u)  satisfies  the  dual  problem 
and  also 

/o(x)  =  *Kx,  u).  (2.12) 

Proof  See  Mangasarin.18 

The  reader  is  referred  to  Mangasarin  (Ref.  16,  p.  105)  for  a  discussion  of  suitable  constraint 
qualification  conditions. 


3.  PROPERTIES  OF  AUGMENTED  COST  FUNCTION 


In  this  section  we  examine  some  of  the  properties  of  the  augmented  cost  function  Qt{x). 
Under  the  assumptions  given  in  Section  2,  and  including  the  assumption  that  problem 
MP  is  a  convex  programming  problem,  Lootsma1  has  shown  that  {M*)  has  the  following 
properties: 

(a)  for  every  t  there  exists  an  x(t)e  Rn  minimizing  0<(x), 

(b)  if  /  <  v  then  /  <fo(x(t))  ^  v  and  Qt(x(t))  >  0,  and 

(c)  if  /  ^  v  then  (><(x(0)  =  0- 

In  addition  we  need  the  following  results  shown  by  Lootsma. 

THEOREM  3.1.  Suppose /<(■)  e  C1(Rn ;  R1),  i  =  0,  1, . .  M  m  and  that  problem  MP  is  a  convex 
programming  problem.  Then  for  every  K  v  an  admissible  solution  of  problem  DP  is  given 
by  WO* «(/))  where  x(t)  minimizes  Qt(x)  over  Rn  and  u(t)  is  given  by 


max  [0,  fi(x(t ))]  . 

-  Mm)-'  . “ 


(3.1) 


Proof  This  may  be  shown  by  direct  substitution. 

THEOREM  3.2.  Suppose  /<(•)  e  C2(R*;  R1),  i  =  0,  I, . . .,  m,  problem  MP  is  a  convex  pro¬ 
gramming  problem  and  there  exists  a  Kuhn-Tucker  point  (x,  u)  which  satisfies  the  Jacobian 
uniqueness  condition,  then 

lim  (x(f),  w(f))  =  (x,  u).  (32) 

9 

t  <  v 

Proof.  This  may  be  shown  by  making  the  obvious  modifications  to  the  proof  given  in  Lootsma 
(Ref.  I,  p.  38). 

LEMMA  3.1.  Suppose /i(  )  e  C2(R";  Rl),  i  =  0.  I . m,  problem  MP  is  a  convex  program¬ 

ming  problem  and  these  exists  a  Kuhn-Tucker  point  y  =  (x,  u)e  Rnx  Rm  which  satisfies  the 
Jacobian  uniqueness  condition,  then  for  t  <  ? 

MO  =  9  -  W;  O  ’  W.  w  -  v)  +  oi(f  -  m 


(3.3) 


where 


f(y\ t)  = 


foTz(x)  +  2  U(/tl 
i  1 

max  (0,/i(jf)) 

uifi(x)  -  -  />(*) 

Mx)  -  / 


(3.4) 


../.«>  -  T.<y-<')v^) 

/<>(*)  -  t 

andjKO  =  WO.  40 )• 

Proo/  To  investigate  the  behaviour  of  (*(/),  40)  for  /  in  a  neighbourhood  of  v,  we  note  from 
the  definition  of  *(/)  and  (2.4)  that  for  t  ^  v 

fo*{x(t))[n lax  (0,/o(*(0)  -  0]  +  2/<*(*(0)[max  (0, /((*(/)))]  =  0. 

i  ~  i 

From  theorem  3.1  this  is  equivalent  to 

/o*(40)  +  £  Ui(t)Mx(t))  =  0, 


/=  i 

-  ft '  - » . * 


(3.5) 

(3.6) 

(3.7) 


for  /  ^  v.  Equations  (3.6)  and  (3.7)  implicitly  define  (jt(f),  40)  m  terms  of  the  parameter  t. 

Without  loss  of  generality  assume  that  the  set  of  active  constraints  I{x)  =  {1*.  •  •>*}• 
From  theorem  2. 1  and  the  Jacobian  uniqueness  conditions  it  follows  that 


fi(x)  =  0  and  u*  >  0  for  i  =  I, . . 


and 


(3*8) 

(3.9) 


/<(x)  <  0  and  iit  =  0  for  i  =  a  +  1, . . m. 

The  continuity  of  the  functions  fi(x)s  /  =  I, . . iff,  and  the  results  of  theorems  3.1  and  3.2 
lead  to  the  conclusion  that  there  exists  an  e  >  0  so  that  for  /  e  [P  —  e,  v), 

fi(x(t))  >  0  and  w<(0  >0  for  i  =  1, . . (3.10) 

and 

fi{x(t))  <  0  and  w*(0  =  0  for  i  =  «  +  1, . . w.  (3. II) 

From  (3.4)  it  can  be  seen  that  F(x,  u;  v)  =  0,  and  also  that  the  matrix  partial  derivative  of  F 
with  respect  to  y  is 


F^y;  0  = 


/>2(*,  u) 


Ulfizix) 


Uaf*i(x) 


2(/o  —  OA/i*  —  fifox 
(/o-02 


2(/o  —  Of  a  fax  fzfox 

(fo  -  O2 


O 


A% ...  ■  fl 


o 


f 2 


+  lx 


•  7  m 


/«  +  l 


fm 


.  (3*12) 
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It  follows  from  the  argument  given  by  Lootsma  (Ref.  I;  p.  17)  that  the  matrix  F^y\  v)  is 
non-singular.  By  the  implicit  function  theorem12  there  exists  a  neighbourhood,  of  v  and  a 
unique  function  y(t)  e  C%N;  R**m)  such  that  y(v)  =  y ,  and  F(y(t);  /)  =  0  for  t  e  N. 

From  Taylor's  theorem  and  noting  that 


we  have 


_  +  W;  *)  =  0, 

at  [i=t, 

y(t)  =  y  +  Fy(y;  v)~1Ft(y;  v)(r  —  v)  +  Ol(t  -  v)2]. 


LEMMA  3.2.  Suppose  /<(  )  e  C2(RW;  R1),  i  =  0,  I, . . /w,  problem  MP  is  a  convex  program¬ 
ming  problem  and  there  exists  a  Kuhn-Tucker  point  (x,  u )  which  satisfies  the  Jacobian  unique¬ 
ness  condition,  then 

^/oWO)  .  = 

at  t^v~  I  ~f-  ft 


*  '(*  WM;  f)  , 

+  .  '  V)F((^;  P))TZ>2(.f,  u)(EiFv  >(>N  P)/^:  P)h 

i  -+-  p 

where 

0  =  2  w<2,  and  the  wx(w|w)  matrix  E\  and  the  mx(m-hn)  matrix  Et  are  defined  as 

i  €  /(*) 

Ei  =  [/  j  0]  and  £2  -  [0  ;  /], 

respectively. 

Proof.  From  the  assumptions  given  above  it  follows  that  (3.10)  and  (3.(1)  hold  for  /€/(£) 
and  i  $  7(x),  respectively. 

Considering  the  expression 

/oWO)+  2  Uifi(x{t)),  (3.15) 

I  e  m 

and  expanding  it  into  a  Taylor  series  we  have 

/oMO)  +  2  QtMx(t))  =/o(x)  +  2  s</<(v)  +  f/.^)  +  2  «(/<*(*)](*(')  -  •?)  f 
i  e  /(jt)  /  e  #(jf)  L  /  e  /(*)  J 

+  1WO  -  X)TD*(xy  QXx{t)  -x)  +  y(jc,  x(/))||x(0  -  Jc||2,  (3.16) 

where  y(x,x(t))  ->0  as  *(/)  ->  x,  and  ||*||  denotes  the  Euclidean  norm  on  Rn .  Consequently 
from  (3.10),  (3.11),  equation  (3.16)  and  theorems  2.1  and  3.1  we  have 

Mx(t))  +  (Mx(t))  -  t)  2  umU)^ 

/€  /(Jf) 

=^Mx)  +  1W/)  -  x)D*(x,  uX^U)  -  x)  +  y(xf  x(t))  ||*(/)  -  x  ||2.  (3.17) 

Defining  0(/)  =  2  w<w<(r),  (3.17)  becomes 

ie  Hx) 

/.WO)  =  /o(f)  1  |  -  *>  +  2(l  in,)!*1'*  ~  *)**<*> “H40  -  f)  t 

+ *’•'« 

Let  us  define  the  ffx(/»f/w)  matrix  £1  and  the  mx(n+m)  matrix  E%  as 

Ei  =  [/  j  0]  and  £2  -  [0  j  /], 
respectively.  From  lemma  3. 1  we  have 


*(0  -  x  =  ~E\F¥  Hy;  v)  Ft{y\  v){t  -  v)  4-  0[{t  -  v)2]. 
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so  that 


m 


Mx(t))  =/o(.f)  +  j  ”’^  (/  -  P) 

+  2(i  +  *0;  ?)  W;  p»T/)2(-f,  Hi5;  *)  W;  v)X‘  -  *)2  +  <?[(/  -  p>*]. 


Differentiating  (3.20)  with  respect  to  t ,  we  find 


~/»W0) 


=  ■■■■-,  and 
\t=V-  1  "I"  p 


c/2 

J/2 


/oWO) 


2^£2Ftf  H^;v)  W;  v) 

(1  +  0)2 


+ 


(3.20) 

(321) 

(3.22) 


+  ,,  ,  -.(£i/yl(j;  V)  Ft(y  \  v))TD2{x ,  v)  £/(>;  v)). 

(i  ~h  p) 

THEOREM  3.3.  Suppose  /<(  •)  e  C2(Rn ;  Z?1),  i  =  0,  1, . . m,  problem  MP  is  a  convex  pro¬ 
gramming  problem,  and  there  exists  a  Kuhn-Tucker  point  (Jc,  ii)  which  satisfies  the  Jacobian 
uniqueness  conditions.  Then  Qt(x(t))  has  a  second  order  zero  at  /  =  v. 

Proof \  Considering  £><(*(/))  we  find 


j'Qt WO)  =  2  max  (0,/O(jf(0)  -  l)(^*  fo*(x(t))  -  1  j  +  2  2 
Since  x(t)  is  a  stationary  point  of  £?*(*(/))  we  have 


max  [0Ji(x{t))}^fiz(x(t)). 


so  that 


max  (0,Mx(t))  -  i)foAx(t))  +  S  max  [0, /,(*(/ ))]/(*(*(' ))  =  0, 

I  I 


(?f(*(0)  =  -2  max  (0,/o(x(0)  -  0- 


(3.23) 

(3.24) 

(3.25) 


Now  from  theorem  3.2  as  t  ->  v  (t  <  v)  then  x(/)  converges  to  .f,  so  that 

d 


dt 


Qt(x(i)) 


From  (3.25)  we  have  for  f  <.  v  that 

d 


=  0. 


Thus 


d 2 
dt2 


dt 


GiMO) 


£<(*('))  -  -2(/0(jr(O)  -  /). 


- 


_i) 


(3.26) 

(3.27) 


2 

”  1  +  P 

Since  ^  ^  0  it  follows  that  the  right  hand  side  of  (3.27)  is  always  non-zero,  and  as  a  consequence 
£M*(0)  has  a  second  order  zero  at  t  =&  v. 


4.  LOCAL  CONVERGENCE  BEHAVIOUR  OF  ITERATION  ALGORITHMS 

In  this  section  we  examine  the  local  rate  of  convergence  for  a  number  of  iterative  algorithms 
which  adjust  the  parameter  /  of  function  Qt(x)  so  that  the  sequence  1,2,...  converges 

to  v,  and  the  corresponding  sequence  *(/<)  converges  to  x ,  the  optimum  solution  of  problem  MP. 
Because  the  function  (?*(■*(/))  has  the  property, 

f  -0  for  all  /  ^  f, 

QtW))<  (4.1) 

(^  >  0  for  all  t  ^  ?, 
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it  has  a  non-unique  minimum.  Consequently  procedures  for  generating  the  sequence  tiJ  =  1,2,... 
must  begin  with  /t  •  v.  It  is  possible  to  demonstrate10  that,  under  appropriate  assumptions, 
the  sequences  /*,  /  =■  1,2,...  generated  by  the  procedures  discussed  below  are  monotonic 
increasing,  satisfy  the  inequalities 

ti  ■  /m  ^  \\  (4.2) 

and  converge  to  v. 

As  a  measure  of  the  rate  of  convergence  we  use  the  definition  of  the  order  of  an  iteration 
algorithm, 

ti  i  -  #/#),  i  C  2 .  (w  3) 

due  to  Traub,13  where  ip{t)  will  be  termed  the  iteration  function. 

DEFINITION  4.1.  If  /*,  /  —  1,2, ..  .  is  a  sequence  converging  to  f,  generated  by  (4.3),  and 
there  exists  a  p  ^  0  and  a  non  zero  constant  C  such  that 

ti\  v  „ 


then  p  and  C  are  termed  the  order  of  the  iteration  function  and  the  asymptotic  error  constant 
respectively. 

The  following  theorem  proved  in  Traub13  will  be  used  in  the  sequel. 

THEOREM  4.1.  If  there  exists  a  neighbourhood,  A,  of  v  such  that  </<(  *)  6  0(A;  Rl)  then  0  is 
of  order  p  if  and  only  if 


^(v)  =  v,  ^\v)  —  0,  j  —  1 1 
In  addition  the  asymptotic  error  constant  is  given  by 


and  ^‘(v)  ^  0. 


ti  -  v  i> 


4.1  The  Morrison  Algorithm 


In  reference  6,  Morrison  proposed  the  iteration  function 

+{t)  =  t  •  \  Qti.W))  (4.6) 

for  adjusting  the  parameter  t  in  the  method  of  centres.  It  will  be  observed  that  v)  =  \\  Also 
from  (4.6),  theorem  3.1  and  lemma  3.2,  we  find  that 


f  n(i-)  -  I 


where  p  is  defined  in  lemma  3.2.  Providing  the  set  of  active  constraints  /(.v)  is  non-empty  it 
follows  from  theorem  4,1  that  the  Morrison  algorithm  has  order  p  —  !,  and  so  is  linearly 
convergent.  Furthermore  the  asymptotic  error  constant  is  given  by  (4.7).  These  results  concur 
with  those  derived  by  Lootsma10  by  different  means. 

In  the  case  where  7(.v)  =  that  is  where  problem  MP  has  an  unconstrained  minimum 
in  C,  the  method  of  centres  using  the  Morrison  algorithm  converges  in  one  step. 


4.2  The  Staha  Algorithm 

Staha14  considered  the  application  of  the  Newton-Raphson  root  finding  procedure  to  the 
method  of  centres.  In  this  case  the  iteration  function  is 


VHO  -  f 


QtiW)) 
Qti.W )) 


which  from  (3.26)  becomes 


#,>  ,  :  . 

AMO) -i 


Since  Qt(x(t ))  has  a  second  order  zero  at  /  =  P,  it  follows  from  Traub  (Ref.  13,  p.  25)  that 
(4.9)  is  only  linearly  convergent.  Because  of  the  insight  gained  into  the  operation  of  the  tangent 
method  we  propose  to  show  this  result  for  a  slightly  more  general  case. 

Supposing  that  Qt(x(r))  has  an  m'th  order  zero,  where  m  >  1,  at  /  =  P,  we  can  write 

QAxit))  =  (/  -  v)^g(t)  (4.10) 

where  it  is  assumed  that  g(v)  ^  0.  From  (4.9)  observe  that  i/<v)  =  v.  By  direct  differentiation 
of  (4.8)  and  (4.10)  and  appropriate  substitutions  we  find  that 


Qt(x(t))\ 


1 


QiWt))] 


(t  -  V -  P)m  2[w(w  -  l)g(/)  -f 
(r  -  v)2m  2[mg(t )  -r  .  .  .]2 


and  since  m  --  2  this  expression  simplifies  to 


at  t  =  v.  Consequently  by  Theorem  4.1,  we  conclude  that  (4.9)  is  linearly  convergent  with  an 
asymptotic  error  constant  of  L 

A  comparison  of  (4.11)  and  (4.15)  gives  some  insight  into  the  reason  why  the  tangent 
method,  discussed  in  the  next  section,  is  quadratically  convergent. 

4.j  The  Tangent  Method 

The  tangent  method  was  first  considered  by  Kowalik  et  al.~*  following  a  brief  remark  by 
Morrison6  where  he  deduced  its  form  on  the  basis  of  geometrical  arguments.  In  this  case  the 
iteration  function  is  given  by 

and  only  differs  from  the  Newton  formula  (4.9)  by  a  factor  of  Lootsma10  has  observed  that 
(4.13)  is  super-Iinearly  convergent. 

We  treat  the  general  case  of  Qt(x(t))  having  an  w'th  order  zero  at  /  =  P,  where  m  >  I. 
Referring  to  equation  (4.8),  suppose  that  the  second  term  is  multiplied  by  m,  so  that  the  new 
iteration  function  is 

,  Qt(x(i)) 


ip{t)  =  /  —  m- 


,Q'(x{t)) 


In  this  case  it  is  easy  to  see  ^(v)  =  v,  and  by  analogy  with  (4.1 1)  it  follows  that 

......  .  ({?«' 1 ’(-*-(/ ))]2  (/  -  v)mg(t)(t  -  v)m  2[m(m  -  \)g(t)  -  . . 

=  1  —  tn  v  m 

[<?tnV('»]2  (/  -  V)2m  HngV)  +  -  .  -J* 

Consequently, 

0a,(v)  =  1  —  m  :  w*W  —  0, 
m 

for  every  m  >  1,  and  in  addition 


cP 

tv 

d 2 

tv 

where  is  given  by  (3.27),  and 

at 1  /  p 

<P  „  I 

.,30'WO)  = 


with  the  derivative  on  the  right  hand  side  being  given  in  lemma  3.2. 


Recalling,  Qt(x(t))  has  a  second  order  zero  at  /  =  P,  it  follows  from  theorem  4.1  that  (4.13) 
is  quadratically  convergent,  and  the  asymptotic  error  coefficient  is  given  by  ^*2,(v)/2. 

From  the  above  analysis  we  are  able  to  draw  slightly  stronger  conclusions  than  Lootsma10; 
namely  that  the  tangent  method  is  quadratically  convergent.  In  addition  we  can  see  that  in  the 
more  general  case  where  £*(*(/))  has  an  arbitrary  rrt  th  order  zero  at  t  —  v,  the  more  general 
algorithm  (4.14)  is  always  quadratically  convergent.  The  tangent  method  can  be  viewed  as  an 
application  of  the  generalized  Newton  formula  (4.14)  whose  discovery,  together  with  its  unique 
property,  Traub13  attributes  to  Schroder,  who  reported  it  about  1870. 


4.4  The  Interpolation  Polynomial  Method 

Numerical  experience10  with  the  tangent  method  has  shown  that  it  tends  to  overshoot  the 
minimum  value  v  quite  readily  for  some  problems,  which  causes  the  algorithm  to  fail  unless 
some  ad  hoc  procedure  such  as  reverting  to  the  Morrison  algorithm  is  used.  This  has  led  the 
author  to  formulate  further  super-linearly  convergent  methods  which  may  have  more  stable 
convergence  behaviour  in  a  finite  neighbourhood  of  the  minimum  point  v. 

We  now  show  that  algorithms  based  on  the  quadratic  Newton  interpolation  formula  are 
super-linearly  convergent.  Only  an  outline,  which  is  sufficient  for  our  purposes,  of  the  algorithms 
will  be  given.  A  detailed  discussion,  including  a  consideration  of  suitable  starting  procedures, 
will  be  the  subject  of  a  further  report. 

To  facilitate  the  discussion  the  following  notation  is  introduced.  The  function  S(t)  is  defined 
as  S(t)  =  V for  all  real  values  of  t.  In  addition  supposing  that  t  =  ti  we  define  Si  =  S(/<) 
and  Qt  —  Qn(x(ti)).  The  quadratic  Newton  interpolation  polynomial  for  the  function  S(f)  at 
the  points  ft ,  /<-i,  tt- 2  with  corresponding  function  values  Su  5*-i,  Si- 2  is  given  by 

Ps(t)  =  +  (t  -  U)S(tu  ti  i)  t  it  -  n)(t  -  U-i )S(tu  U  u  U-il  (4.19) 

where  the  divided  difference  operators  are  defined  by 

c  _  c 

S(ti ,  ti  1)  =  *  \  and  (4.20) 

ti  —  ti  1 


S(ti,  ti- 1,  ti- 2) 


*S(fj,  h  1) 
U 


S(U-u  ti  2) 
ti  2 


(4.21) 


The  polynomial  PqU)  is  defined  in  a  similar  manner  using  the  points  0,  fr  1,  2  and  the 

function  values  Qi ,  Qi-u  Qt  2- 


Algorithm  A 


Step  0.  Select  /o,  tu  t>  such  that  to  <  ti  <  t  >  <  f\  and  set  /  =  2. 
Step  1.  Compute  the  solutions  Si  )  given  by 


Step  2.  Find  /a/,  where 


St  j  =  min  S(ti  }\  0,1, 2. 

X  6  R« 

f  _ 1 _  S(ti%  ti  1) 

2  2S(/*,  ti  1,  ti  2) 


Step  3.  If  Ps(t\f)  ^  0  set  tt+\  =  tu. 
If  Ps(t\t)  <  0  then  set 


(4.22) 

(4.23) 


ti  u  =  ?  st 


jtsi2  —  (Si  —  S(ti ,  ti  i)/<  ■+*  S(ti ,  ti-u  ti  2)ti  ti  1) 

5(tf,  ti  1,  ti- 2) 


(4.24) 


Step  4.  If  5f+i  =  0  set  =  and  stop;  otherwise  increment  parameter  i  and  return  to 

step  I. 


Algorithm  B 

This  is  essentially  the  same  as  algorithm  A  excepting  that  function  (?t(.v(/))  replaces  S(f) 
and  polynomial  Pq(t)  replaces  Ps(t). 
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Of  course  the  iteration  function  for  each  of  these  algorithms  is  no  longer  a  simple  expression. 
However  it  should  be  observed  that  in  a  sufficiently  small  neighbourhood  of  v,  the  function 
can  be  written  as 

QMO)  =*  1  -(/  -  P)2  +  Ol(t  -  v)3].  (4.25) 

1  tj 

Consequently  Q(tt ,  ft- j,  ft  2)  >  0  which  implies  that  /a/  is  the  minimum  of  a  quadratic  poly¬ 
nomial.  A  similar  argument  applies  for  function  5(0.  In  addition  the  following  lemma  shows 
that  in  a  sufficiently  small  neighbourhood  of  v,  the  polynomial  Ps{t)  (or  /VO)  has  a  real  zero 
for  t  <  V,  so  that  for  this  case  the  iteration  function,  which  will  be  denoted  by  ips(t 
is  given  by  (4.24). 

LEMMA  4.1.  For  a  sufficiently  small  €  >  0  suppose  513,(/)  [or  £V3I(*(0)]  is  continuous  and 
non-zero  on  the  interval  (v  —  c,  v).  Let  the  points  ft  2,  ft  u  ft  e  (v  —  e,  v)  satisfy  the  conditions 
ti—'i  <  ti -1  <  ft  <  v.  If  5<3)(v)  ^  A  >  0,  (0t(3,(v)  >  A  >  0),  then  Ps(t)  (Pq( ())  has  a  real  root 
ftr 1  such  that  ft  <  ft*i  <  v,  so  that  the  sequence  ft,  #  =  1,2,...  is  monotone  increasing. 

Proof.  Since  5<3)(v)  ^  A  >  0  it  follows  from  continuity  that  there  exists  an  €  >  0  such  that 
5(3)(f)  >  0  for  /  g  (v  —  e,  v).  We  take  ft  >,  ft-i,  ft  e  (v  —  e,  v).  The  error  equation  for  the  inter¬ 
polation  polynomial  Ps(t)  is  given  by  Reference  15, 

j  0 

where  £(/)  lies  in  the  interval  (v  —  e,  v).  Consequently 

p  ...  S'*W))  T"S\. 

w  =  -  3!  [  [0 

j  0 

and  from  the  result  that  5(3)(/)  >  0,  we  have  Ps(v)  <  0.  In  addition  from  property  (6)  in 
Section  3  we  see  that  Ps(ti)  >  0.  From  the  continuity  of  P$(t )  it  follows  that  it  possesses  a 
real  zero  ft+i  in  the  interval  ft  <  ft+i  <  P,  and  that  the  sequence  ft,  /  =  1,2,...  is  monotone 
increasing. 

THEOREM  4.2.  Suppose  /((*)  e  C2(Rn;  R1),  /  =  0,  1, . . m,  problem  MP  is  a  convex  pro¬ 
gramming  problem,  and  there  exists  a  Kuhn-Tucker  point  (.v,  u)  which  satisfies  the  Jacobian 
uniqueness  conditions.  In  addition,  for  Algorithms  A  and  B  suppose  that  5<3>(v)  ^  A  >  0,  and 

(?t(3)(*(0)  ^  A  >  0  respectively.  Then  the  iteration  function  tft.s(f)  has  an  order  p  21  1*84 

t=v 

with  an  approximate  asymptotic  error  coefficient 

S' 3)(P)  l0  42 

3!  5u>(v) J  * 

and  the  iteration  function  )  has  an  order  p  ^  1-26,  with  an  approximate  asymptotic  error 
coefficient 

I  "I0-26 

.  QflKxiO),  _ 

1  t~=v 

3  Q,' "{*«)) , 

L  f  r  J 

Proof.  We  restrict  consideration  to  Algorithm  A,  as  the  proof  for  Algorithm  B  follows  along 
analogous  lines. 

From  lemma  4,1  it  follows  that  in  a  sufficiently  small  neighbourhood  of  v  the  sequence  of 
parameters  ft,  i  =  1,2,...  are  determined  by  (4.24),  which  gives  a  root  of  the  polynomial  Ps(t). 
In  addition  from  theorem  3.3  and  the  definition  of  5(f)  we  note  that  5(f)  has  a  zero  of  multiplicity 
one  at  v.  Traub  (Ref.  13,  Theorem  7.5)  has  shown  that  the  order,  p,  of  such  an  algorithm  is 
given  by  the  unique  real  positive  root  of 

,  |  |)  0,  (4.28) 


-  Oh  (4.26) 


-  Oh  (4.27) 
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and  the  asymptotic  error  coefficient  is 

[m\  5,3,(  »•)!''  l"m  1‘ 

3!  4y«'*><P)J 

where  m  is  the  multiplicity  of  the  zero  of  5(f)  at  v.  In  this  case  m  1,  and  consequent^  the 
order  p  ^  I  *  84. 

5.  CONCLUDING  REMARKS 

In  this  report  the  convergence  of  a  number  of  iterative  procedures  for  method  of  centres 
algorithms  has  been  investigated.  The  tangent  method,  which  was  previously  known  to  be 
super-linearly  convergent,  has  been  shown  to  be  quadratically  convergent,  and  the  known 
results  for  the  Staha  and  Morrison  methods  were  proved  as  an  easy  consequence  of  the  present 
approach.  In  addition,  the  rate  of  convergence  for  two  quadratic  interpolation  pol>nomial 
methods  developed  by  the  author  was  studied.  It  was  shown  that  both  methods  are  super- 
linearly  convergent,  with  Algorithm  A,  in  which  the  function  is  transformed  into  a 

function  having  a  simple  zero,  being  almost  quadratically  convergent.  Limited  numerical 
experience  with  Algorithm  B,  where  the  function  0,(.v(r))  is  interpolated  directly,  has  shown 
it  to  perform  very  favourably  in  comparison  with  the  tangent  method.  It  is  proposed  to  extend 
these  comparisons  in  the  light  of  the  above  theoretical  results. 

The  key  to  the  main  results  obtained  in  this  work  has  been  the  demonstration  that  the 
function  £)*(*(/ ))  has  a  zero  of  multiplicity  two  at  the  point  t  f.  While  (M.v)  has  the  special 
form  given  by  (2.4),  it  appears  that  similar  results  concerning  the  multiplicity  of  the  zero  of 
@f(x(/))  can  be  obtained  for  a  generalization  of  (2.4),  provided,  suitable  differentiability 
assumptions  are  made. 

Apart  from  the  concrete  results  obtained  above,  the  insight  gained  about  the  analytic 
properties  of  Qt(x(t))  leads  us  to  speculate  on  suitable  alternative  algorithms  for  finding  the 
zeros  of  this  function.  Such  methods  as  inverse  interpolation  and  Halley's  method,13  as  well 
as  other  methods  of  transforming  (?*(*(/))  into  a  function  having  a  zero  of  multiplicity  one  are 
worthy  of  further  consideration. 


REFERENCES 


1.  Lootsma,  F.  A.,  “Boundary  Properties  of  Penalty  Functions  for  Constrained  Minimization". 
Philips  Res.  Rep.  Suppl .,  No.  3,  1970. 

2.  Huard,  P.,  “Resolution  of  Mathematical  Programming  with  Non-linear  Constraints  b\  the 
Method  of  Centers",  in  Non-linear  Programming ,  (ed.)  J.  Adabie,  North  Holland  Publ.  Co.. 
Amsterdam  (1967),  pp.  207  219. 

3.  Faure,  P.,  Huard,  P.,  “Resultals  Nouveaux  Relatif  a  la  Methode  des  Centres'*.  Paper 
presented  at  Fourth  Int.  Con/,  on  Operations  Res. ,  Boston,  Mass.,  1966. 

4.  Lootsma,  F.  A.,  “Constrained  Optimization  via  Parameter- Free  Penults  Functions", 
Philips  Res.  Rep.,  23,  (1968),  pp.  424^437. 

5.  Pironneau,  O.,  Polak,  E..  "On  the  Rate  of  Convergence  of  Certain  Methods  of  Centers", 
Math.  Programming ,  2  (1972),  pp.  230-257. 

6.  Morrison,  D.  D.,  "Optimization  by  Least  Squares".  SIAM  J  Sumer.  Anal.,  5  (1968), 

pp.  83-88. 

7.  Kowalik,  J.,  et  a /.,  "A  New  Method  for  Constrained  Optimization  Problems".  Operations 
Res.,  17  (1969),  pp.  973-983. 

8.  Kowalik,  J.,  “Nonlinear  Programming  Procedures  and  Design  Optimization",  Acta  Poly- 
technica  Scandinavica ,  Trondheim,  13  (1966). 

9.  Staha,  R.  L.,  and  Himmelblau.  D  M.,  “Evaluation  of  Constrained  Nonlinear  Programming 
Techniques",  Harking  paper ,  Lniv.  of  Texas  (Austin,  Texas)  1973. 

10.  Lootsma,  F.  A.,  “Convergence  Rates  of  Quadratic  Exterior  Penalty-Function  Methods  for 
Solving  Constrained  Minimization  Problems".  Philips  Res  Rep  ,  29  (1974).  pp.  I  12 

11.  Wolfe,  P.,  “A  Duality  Theorem  for  Non-linear  Programming".  Quarterly  of  Applied 
Mathematics ,  19  (1961)  pp.  239  244. 

12.  Bliss,  J.  A.,  Lectures  on  the  Calculus  of  Variations,  Phoenix  Scientific  Series,  The  University 
of  Chicago  Press,  Chicago,  1961. 

13.  Traub,  J.  F.,  Iterative  Methods  for  the  Solution  of  Equations,  Prentice-Hall.  Inc.,  Englewood 
Cliffs,  N.J.,  1964. 

14.  Staha,  R.  L.,  Ph  D.  Dissertation,  Univ.  of  Texas  (Austin,  Texas),  1972. 

15.  Hildebrand,  F.  B.,  Introduction  to  Numerical  Analysis,  McGraw-Hill  Book  Company,  Inc  . 
New  York,  1956. 


f 


16.  Mangasarian.O.  L Nonlinear  Programming,  McGraw-Hill  Book  Company,  Inc.,  New  York, 
1969. 


DISTRIBUTION 


Copy  No. 


AUSTRALIA 

DefirtMeK  of  Defence 

Ccttril  OAcc 
C  hicf  Defence  Scientist 
Deputy  Chief  Defence  Scientist 
Superintendent,  Science  and  Technology  Programs 
Defence  Library 

Australian  L*  ’nee  Science  and  Technical  Representative  (U  K.) 
Counsellor,  Defence  Science  (U  S  A.) 

Assistant  Secretary,  D.I.S.B. 

Joint  Intelligence  Organisation 

Aeronautical  Research  Laboratories 

C  hief  Superintendent 
Library 

Superintendent  Systems  Division 
Divisional  Kile  Systems 
Author  Ronald  B.  Zmood 

Materials  Research  Laboratories 

Library 

Defence  Research  C  entre,  SaBsbory 

Library 

C  entral  Staples  Establishment 

Information  C  entre 

Engineering  Development  C  entre 

Library 

Air  Force  Office 

RAAK  Acadcmv,  Point  Cook 

Army  ORke 

Royal  Military  (  ollegc  Library 

Deportment  of  Transport 

Secretary  Library 

Statatory,  State  AatborRies  an 4  Inins try 

Qantas,  Library 

Ansett  Airlines  of  Australia,  Library 

Australian  Paper  Manufacturers,  Dr  Norman 

BHP,  Melbourne  Research  Laboratories 

CSIRO  Applied  Physics  Division,  Library 

Hawker  de  Havilland  Pty  Ltd,  Librarian,  Bankstown 

ICI  Australia  Ltd.  Library 

Australian  Road  Research  Board 


1 

2 

3 

4 


5-21 

22 


23 

24 

25 

26 

27 

28 

29 

30 

31 

32 

33 


34  35 


36 

37 

38 

39 

40 

41 

42 

43 


u 


Universities  and  Colleges 

Adelaide  Barr  Smith  Library  44 

Flinders  Library  45 

James  Cook  Library  46 

La  Trobc  Library  47 

Melbourne  Engineering  Library  48 

Monash  Library  59 

Newcastle  Library  50 

New  England  Library  51 

Sydney  Engineering  Library  52 

N.S.W.  Physical  Sciences  Library  53 

Queensland  Library  54 

Tasmania  Engineering  Library  55 

Western  Australian  Library  56 

R.M.I.T.  Library  57 

CANADA 

Universities  and  Colleges 

Toronto  Institute  for  Aerospace  Studies  58 

FRANCE 

AGARD,  Library  59 

ONERA,  Library  60 

GERMANY 

ZLDI  61 

INDIA 

Defence  Ministry,  Aero  Development  Establishment,  Library  62 

Indian  Institute  of  Technology,  Library  63 

National  Aeronautical  Laboratory,  Director  64 

ISRAEL 

Technion  Israel  Institute  of  Technology,  Prof.  J.  Singer  65 

ITALY 

Associazione  Italiana  di  Aeronautica  e  Astronautica  66 

JAPAN 

National  Aerospace  Laboratory,  Library  67 

Universities 

Tokyo  Institute  of  Space  and  Aerospace  68 

NETHERLANDS 

National  Aerospace  Laboratory  (NLR),  Library  69 

NEW  ZEALAND 

Air  Department,  R.N.Z.A.F.  Aero  Documents  Section  70 

SWEDEN 

Aeronautical  Research  Institute  71 

Chalmers  Institute  of  Technology,  Library  72 

Research  Institute  of  the  Swedish  National  Defence  73 

SWITZERLAND 

Institute  of  Aerodynamics,  Prof.  J.  Ackerct 


74 


i 


UNITED  KINGDOM 

Aeronautical  Research  Council,  Secretary  75 

Royal  Aircraft  Establishment,  Library,  Farnborough  76 

Royal  Aircraft  Establishment,  Library,  Bedford  77 

National  Physical  Laboratories,  Library  78 

British  Library,  Science  Reference  Library  89 

British  Library,  Lending  Division  80 

Central  Electricity  Generating  Board  81 

Science  Museum  Library  82 

British  Aerospace  Corporation : 

Weybridge-Bristol  Division  83 

Warton  Division  84 

Transport  and  Road  Research  Laboratories  85 

Universities  and  Colleges 

Bristol  Library,  Engineering  Department  86 

Dr  W.  Chester,  Mathematics  Department  87 

Cambridge  Library,  Engineering  Department  88 

Manchester  Professor,  Applied  Mathematics  99 

Nottingham  Library  90 

Southampton  Library  91 

Strathclyde  Library  92 

Cranfield  Institute  Library  93 

of  Technology  Prof.  Lefebvre  94 

Imperial  College  The  Head  95 

Loughborough 

University  Department  of  Mathematics  96 

University 

of  Warwick  Control  Theory  Centre  97 

UNITED  STATES  OF  AMERICA 

NASA  Scientific  and  Technical  Information  Facility  98 

Sandia  Group  Research  Organisation  99 

American  Institute  of  Aeronautics  and  Astronautics  100 

Applied  Mechanics  Review  101 

The  John  Crerar  Library  102 

Boeing  Co.,  Head  Office  103 

Esso  Research  Laboratories,  Director  104 

Lockheed  Missiles  and  Space  Company  105 

Lockheed  California  Company  106 

Lockheed  Georgia  Company  107 

McDonnell  Douglas  Corporation,  Director  118 

Westinghouse  Laboratories,  Director  119 

Calspan  Corporation  (formerly  Cornell  Aero  Labs.)  1 10 

Battelle  Memorial  Institute,  Library  1 1 1 

Universities  and  Colleges 

Harvard  Professor  G.  F.  Carrier,  Applied  Mathematics  1 12 

Stanford  Department  of  Aeronautics  Library  1 1 3 

Polytechnic  Institute 

of  New  York  Aeronautical  Labs.  Library  1 14 

California  Institute 

of  Technology  Graduate  Aeronautical  Labs.  Library  115 

Massachusetts  Institute 

of  Technology  Library  1 1 6 

Michigan  Prof.  E.  Gilbert  117 


Spares 


118-127 


